The Quality of Expertise

Policy-makers and managers often turn to experts when in need of information: because they are more informed than others of the content and quality of current and past research, they should provide the best advice. I show, however, that we should expect experts to be systematically biased, potentially to the point that they are less reliable sources of information than non-experts. This is because the decision to research a question implies a belief that research will be fruitful. If priors about the impact of current work are correct, on average, then those who select into researching a question are optimistic about the quality of current work. In areas that are new, or feature new research technologies (e.g., data sources, technical methods, or paradigms), the selection problem is less important than the bene…t of greater knowledge: experts will indeed be experts. In areas that are old and lack new research technologies, there will be signi…cant bias. Furthermore, consistent with a large body of empirical research, this selection problem implies that experts who express greater con…dence in their beliefs will be, on average, less accurate. This paper provides many empirical implications for expert accuracy, as well as mechanism design implications for hiring, task assignment, and referee assignment. JEL Classi…cations: D80, J44 Keywords: Expertise, Learning, Prediction Policy-makers and managers often need information in order to make a decision, and turn to “experts”for that information. Experts have a deep knowledge of the theory and evidence concerning their subject of expertise, and are trained with the skills to evaluate that evidence. A long theoretical literature in the …eld of economics, beginning with Crawford and Sobel (1982), asks the question of how much information policy-makers are able to elicit from experts, given that experts may not agree with the objectives of the policy-maker and may therefore attempt to convey false information. In this paper, I tackle a question one step primitive to the standard: do unbiased experts even exist and, if so, under what circumstances? Stated another way, even if an expert agrees with the objectives of the policy-maker, in what cases can policy-makers trust that the expert’s information is correct? I show that experts are, on average, biased and may not the best source of information, even if they are the most informed individuals in the economy and genuinely wish to provide accurate information. This fact is consistent with recent empirical work (e.g., Tetlock, 2006). I provide parameters that are, in principle at least, observable to the policy-maker that help her determine what weight to put on expert opinion. Given that there are likely several semantic issues that have already arisen in the reader’s mind, I should de…ne terms up front. For the bulk of this manuscript, I focus on what is, for practical purposes, academic expertise. An expert is somebody who actively researches questions, such as “does smoking cause lung cancer?”, “does de…cit-…nanced …scal stimulus reduce the duration of recessions?”, or “does a transaction tax reduce liquidity in …nancial markets?”These questions fall into …elds, like medicine, economics, or …nance. While the model will be precise in de…ning what constitutes a question or a …eld, in practice it may be more di¢ cult to draw a distinction. Is “the downfall of the Roman Empire” a …eld, in which “what caused it?” is a question? Or is the question “What caused the downfall of the Roman Empire?”while the …eld is Roman History? I will leave